---
title: Create Evaluation Datasets
---

<div style={{
    position: 'relative',
    paddingBottom: '56.25%', // 16:9 aspect ratio
    height: 0,
    overflow: 'hidden',
    maxWidth: '100%',
    marginBottom: '20px'
}}>
    <iframe
        src="https://www.loom.com/embed/c84b219b7c454476bb276abaaf4df777?sid=023a4650-a0cc-46c0-9613-111fc26b020a"
        frameborder="0"
        webkitallowfullscreen
        mozallowfullscreen
        allowfullscreen
        style={{
            position: 'absolute',
            top: 0,
            left: 0,
            width: '100%',
            height: '100%',
        }}
    />
</div>

## Building Datasets for RAG Evaluation

This hands-on video demonstrates [dataset](https://www.comet.com/docs/opik/evaluation/manage_datasets) creation using a practical RAG (Retrieval Augmented Generation) example that compares [OpenAI](https://www.comet.com/docs/opik/integrations/openai) and Google [Gemini](https://www.comet.com/docs/opik/integrations/gemini) models. You'll learn how [evaluation datasets](https://www.comet.com/docs/opik/evaluation/manage_datasets) serve as the foundation for systematic LLM testing - they're collections of example inputs your application will encounter along with expected outputs, similar to validation sets in traditional machine learning.

## Key Highlights

- **Practical RAG Setup**: Complete example showing [OpenAI](https://www.comet.com/docs/opik/integrations/openai) vs Google [Gemini](https://www.comet.com/docs/opik/integrations/gemini) model comparison with vector store integration using [LangChain](https://www.comet.com/docs/opik/integrations/langchain) and Chroma
- **Dual Creation Methods**: Create [datasets](https://www.comet.com/docs/opik/evaluation/manage_datasets) via UI (select [traces](https://www.comet.com/docs/opik/tracing/log_traces) and "add to dataset") or programmatically using the Opik client with [`get_or_create_dataset()`](https://www.comet.com/docs/opik/evaluation/manage_datasets#creating-a-dataset-using-the-sdk)
- **Flexible Data Structure**: Define custom fields in [dataset](https://www.comet.com/docs/opik/evaluation/manage_datasets) items based on your specific use case - make inputs and outputs as verbose as your application needs
- **Automatic Deduplication**: Opik automatically prevents duplicate entries in [datasets](https://www.comet.com/docs/opik/evaluation/manage_datasets), ensuring data quality and consistency
- **Multiple [Dataset](https://www.comet.com/docs/opik/evaluation/manage_datasets) Strategy**: Create focused [datasets](https://www.comet.com/docs/opik/evaluation/manage_datasets) for different aspects - common questions, edge cases, failure modes, and specific capabilities like reasoning or summarization
- **Trace-to-[Dataset](https://www.comet.com/docs/opik/evaluation/manage_datasets) Conversion**: Leverage existing [traces](https://www.comet.com/docs/opik/tracing/log_traces) by filtering high-performing interactions and converting them directly into [evaluation datasets](https://www.comet.com/docs/opik/evaluation/manage_datasets)
- **Validation Set Approach**: [Datasets](https://www.comet.com/docs/opik/evaluation/manage_datasets) function like traditional ML validation sets, providing representative examples for systematic performance assessment
- **Scalable Architecture**: Use class-based setup to handle different model providers with consistent interfaces while maintaining traceability with [`@track`](https://www.comet.com/docs/opik/tracing/log_traces#decorators) decorators
